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BobbyD211: Hey, how was the fishing trip? 
Fred1432: It was great! | couldn't believe the 
size of the big mounth bass that Jimmy caught. 
BobbyD211: Wow! Sounds like it was an 
awesome trip. 

Fred 1432: It sure was! 





AMM) .°a] B 7 u | tink AO 








aA FIGURE 1 


(prior art) 


© © Instant Messages with Sue = 


402 


Did you hear what Bob did) | <@ 408 
yesterday? y 
Pas 


What did you do?! 


Oh man, it was so great. You 
shoulda been there, man! 

















FIGURE 4 


Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page5d of 19 


U.S. Patent Feb. 23, 2010 Sheet 2 of 8 US 7,669,134 B1 











Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page6 of 19 


U.S. Patent Feb. 23, 2010 Sheet 3 of 8 US 7,669,134 B1 





300 Receive request to initiate instant messaging 
session, identify parties, and determine icon 
for each party 



























304 ue 
Generating message —_ 
to other party? Pe 
deliver signal to other party 
indicating message is being 
generated a 306 






Receive message that 
other party is generatin 
a message? es 


310 





Pe 
Produce graphical image of 
message being generated by 
other party 





Received message ae 
~_ from other party? a 





Display message and icon 
adjacent left side of display 
window 


Ses 


~. 


Y Pa genre to other ~ ae 


“Paty complete? a | 


Send message to other party N 
and display message and icon | 
locally adjacent right side of 
display window 




















= . es a 
Instant messaging ~_ » 
316 ~~. session complete? 
a wae 


™~. a 


N, oF 
ON a AG. 
Y 





FIGURE 3 


Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page7 of 19 


U.S. Patent Feb. 23, 2010 Sheet 4 of 8 US 7,669,134 B1 


404 406 


OCC ist esses wi Suef 


402 


408 











FIGURE 5A 





Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Pages of 19 


U.S. Patent Feb. 23, 2010 Sheet 5 of 8 US 7,669,134 B1 


404 406 


@\©O O Instant Messages with Sue 


408 

















FIGURE 5B 


Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page9 of 19 


U.S. Patent Feb. 23, 2010 Sheet 6 of 8 US 7,669,134 B1 





Receive request to initiate instant messaging 


















300 
session, identify parties, and determine icon 
for each party 
302 
304 
Y Generating message 
to other party? 
deliver signal to other party 
indicating message is being N 
generated a 306 
Y Receive message that 
00 other party is generating 
a message? 
Produce graphical image of oe 
message being generated by N 
other party based on time 310 
created 
ba Received message 
602 from other party? 


Display message and icon 
adjacent left side of display 


window based on time 314 
completed 6 Ee 


Y ah Message to shen 














| party complete? ~~ a 
~~ — 
Send message to other party N 
and display message and icon 
locally adjacent right side of 
display window based on time 
completed 
7 a 
\ Ae Instant messaging > N 
604 ci complete? _~ 


SF. 
" 318 


FIGURE 6 


Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page10 of 19 


U.S. Patent Feb. 23, 2010 Sheet 7 of 8 US 7,669,134 B1 


404 





402 


Did you hear what Guiomar coo did 408 
Guiomar coo 





ss 700 
#3 coe 


Oh man, it was so Geren You 
Geren been there, man! 





FIGURE 7 


404 


© O Instant Messages with Sue 


408 








Oh man, it was so great. You 
shoulda been there, man! 


Bre { What did you do?! ) ae 

















FIGURE 8 


Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page11 of 19 


U.S. Patent Feb. 23, 2010 Sheet 8 of 8 US 7,669,134 B1 





The balloon will fit cola 
around this message. 


FIGURE 9B | 


wo 


[LYS 


= 
WwW 





FIGURE 10A 


1000 


height-(9+13) 


width-(20+12) 


FIGURE 10B 


Case5:11-cv-01846-LHK Documenti-2 Filed04/15/11 Page12 of 19 


US 7,669,134 B1 


1 


METHOD AND APPARATUS FOR 
DISPLAYING INFORMATION DURING AN 
INSTANT MESSAGING SESSION 


BACKGROUND OF THE INVENTION 


1. Field of the Invention 

This invention relates generally to a user interface for dis- 
playing an exchange of messages during an instant messaging 
session, and, more particularly, to a method and apparatus for 
displaying instant message exchanges in a manner that 
graphically differentiates the participants in a conversation. 

2. Description of the Related Art 

Networks, such as the Internet, intranets, or other private or 
public networks, are ubiquitous. In fact, many computers are 
connected to one or more networks at the same time. For 
example, a business may have hundreds or even thousands of 
computers coupled to its own private network, which was, at 
least initially, used primarily for storage and exchange of 
computer files. At least some of these same business comput- 
ers may also be coupled to the internet. Further, with the 
development of wireléss devices, ad hoc networks may also 
be formed with properly configured portable devices. Even 
telephonic devices, such as cellular phones, pagers and the 
like, may be coupled to one or more of these networks. Small 
businesses and homes are also often connected in similar 
arrangements. 

All of this connectivity has naturally led to communica- 
tions between various users over these networks. For 
example, electronic mail (e-mail), because of its usefulness, 
is now commonplace. E-mail is now widely used by busi- 
nesses and individuals, and in at least some instances has 
replaced more traditional forms of communications, such as 
mailed letters, facsimiles, telexes, and the like. However, 
e-mail has proven to be somewhat awkward when used to 
carry on an ongoing conversation. 

Instant messaging, on the other hand, allows two or more 
users connected through these networks to carry on an inter- 
active conversation. Exemplary instant messaging systems 
include Apple iChat. AOL Instant Messenger. Microsoft 
MSN Messenger, and the like. ‘lypically. two or more users 
type in messages or select icons, which they send to one 
another. ‘The receiving party(ies) may immediately respond 
with an appropriate message or icon. ‘hese instant messages 
are commonly all displayed in serial fashion, such as shown in 
FIG. 1, usually scrolling the user’s screen from top to bottom. 
Commonly, each message is preceded by a label, such as 
BobbyD211 and Fred1432 in FIG. 1, indicating the identity 
of the author of the message. Heretofore, users have relied on 


these labels, or other limited indicia, to locate and identify 5 


messages from a particular party. Accordingly, it will be 
appreciated that the presentation of each message in substan- 
ually similar format makes it difficult to readily determine the 
authorship of one or more previous messages. Likewise. it is 


difficult to go back and quickly locate a previous message 5: 


without reading through many previous messages. 
The present invention is directed to overcoming or at least 
reducing one or more of the problems set forth above. 


SUMMARY OF THE INVENTION 


. Inone aspect of the present invention, a method is provided 
for displaying information related to a communication ses- 
sion. Information relating to data produced by a first partici- 
pant to the communication session is displayed on a first 
display unit, wherein the information produced by the first 
participant is displayed at a first position on the first display 
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wn 


35 


an 


45 


60 
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unit. Data is received from a second participant to the com- 
munication session, and information relating to the data 
received from the second participant is displayed on the first 
display unit, wherein the information received from the sec- 
ond participant is displayed at a second position on the first 
display unit. The first and second positions are spatially dis- 
tinct. 

In another aspect of the present invention, a computer 
readable program storage device is provided and encoded 
with instructions that, when executed by a computer, per- 
forms a method. The method includes displaying information 
relating to data produced by a first participant to the commu- 
nication session on a first display unit, wherein the informa- 
tion produced by the first participant is displayed at a first 
position on the first display unit. Data is received from a 
second participant to the communication session, and infor- 
mation relating to the data received from the second partici- 
pant is displayed on the first display unit, wherein the infor- 
mation received from the second participant is displayed at a 
second position on the first display unit. The first and second 
positions are spatially distinct. 

In still another aspect of the present invention, a graphical 
user interface for displaying information related to a commu- 
nication session is provided. The interface is comprised of a 
first and a second spatially distinct region. The first region is 
adapted to display at least one message from a first participant 
to the instant messaging session. The second region is 
adapted to display at least one message from a second par- 
ticipant to the instant messaging session, and the first and 


° second spatially distinct regions partially overlap and each 


include at least a portion that is free from overlapping. 

In yet another aspect of the present invention, a method for 
displaying information related to a communication session is 
provided. Information relating to data produced by a partici- 
pant to the communication session is received. The informa- 
tion received from the participant is then at least partially 
displayed within a speech balloon. 

In still another aspect of the present invention, a graphical 
user interface for displaying information related to a commu- 
nication session is provided. The graphical user interface 
comprises a first and second region, wherein the first region is 
adapted to display a speech balloon. The second region is 
adapted to display at least one message from a participant to 
the instant messaging session, wherein the second region is at 
least partially located within the first region. 


BRIET DESCRIPTION OF THE DRAWINGS 


The invention may be understood by reference to the fol- 
lowing description taken in conjunction with the accompany- 
ing drawings, in which like reference numerals identify like 
elements. and in which: 

FIG. } illustrates a view of a screen representative of a 
graphical user interlace of a prior art instant messaging sys- 
tem: 

FIG. 2 illustrates a top-level diagram of onc embodiment of 
a hardware system on which the present invention may be 
implemented; 


FIG. 3 illustrates a flowchart of an embodiment ofa graphi- 


cal user interface that may be executed by components within 
the system of FIG. 1 to produce the exemplary screens of 
FIGS. 4 and 5; 

FIG. 4 illustrates a first view of an exemplary screen rep- 
resentative of a graphical user interface; 

FIGS. 5A-5B illustrate a second and third view of exem- 
plary screens representative of a graphical user interface; 
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FIG. 6 illustrates a flowchart of an alternative embodiment 
ofa graphical user interface that may be executed by compo- 
nents within the system of FIG. 1; 

FIG. 7 illustrates a view of an exemplary screen represen- 
tative of a graphical user interface; 

FIG. 8 illustrates an alternative view of the exemplary 
screen of FIG. 7: 

FIG. 9 illustrates a typical format for a speech bubble; and 

FIG. 10 iflustrates a speech balloon that has been stretched 
or modified to accommodate a message. 

While the invention is susceptible to various modifications 
and alternative forms, specific embodiments thereof have 
been shown by way of example in the drawings andare herein 
described in detail. It should be understood, however, that the 
description herein of specific embodiments is not intended to 
limit the invention to the particular forms disclosed, but on the 
contrary, the intention is to cover all modifications, equiva- 
lents, and alternatives falling within the spirit and scope of the 
invention as defined by the appended claims. 


DETAILED DESCRIPTION OF SPECIFIC 
EMBODIMENTS 


Illustrative embodiments of the invention are described 
below. In the interest of clarity, not all features of an actual 
implementation are described in this specification. It will of 
course be appreciated that in the development of any such 
actual embodiment, numerous implementation-specific deci- 
sions must be made to achieve the developers’ specific goals, 
such as compliance with system-related and business-related 
constraints, which will vary from one implementation to 
another. Moreover, it will be appreciated that such a develop- 
ment effort might be complex and time-consuming, but 
would nevertheless be a routine undertaking for those of 
ordinary skill in the art having the benefit of this disclosure. 

Turning now to FIG. 2, a block diagram depicting a system 
100 in accordance with embodiments of the present invention 
is illustrated. The system 100 includes a plurality of comput- 
ing devices coupled together through one or more network 
connections. For example, a plurality of devices may be 
coupled together via a private or public network, such as a 
Jocal area network (LAN) 102 or the Internet. The actual 
connection between the devices and the LAN 102 may take 
on one or more of any of a variety of forms, such as a network 
interface card (NIC), a modem, a digital subscriber line 
(DSL), a cable modem, a wireless connection, and the like. 
The devices coupled to the LAN 102 may include, for 
example, desktop computers, such as an Apple Macintosh® 
104, a classic Apple Mac® 106, an IBM compatible personal 


computer (PC) 108, and the like. Further, these desktop com- 5 


puters, such as the Apple Macintosh® 104. may be coupled 
together via a smaller sub-I,AN 110, with the sub-LAN 110 
being coupled to the I.AN 102. Portable devices. such as the 
Apple PowerBook ® or iBook: 112, may also be coupled to 


the LAN 102. either directly or as part of the sub-LAN 110. 5: 


Further, other consumer devices, such as cell phones. per- 
sonal data assistants (PDAs), network appliances, and other 
embedded devices may be connected to the LAN 102 so as to 
employ aspects of the instant invention. 

While the invention has been illustrated herein as being 
useful in a network environment, it also has application in 
other connected environments. For example, two or more of 
the devices described above may be coupled together via 
device-to-device connections, such as by hard cabling, radio 
frequency signals (e.g., 802.11(a), 802.11(b), 802.11(g), 
Bluetooth, or the like), infrared coupling, telephone lines and 

modems, or the like. The instant invention may have applica- 
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tion in any environment where two or more users are inter- 
connected and capable of communicating with one another. 

Those skilled in the art will appreciate that network con- 
nections may include a variety of other equipment, such as 
routers, switches, telephone modems, wireless devices, cable 
modems, digital subscriber lines, and the like. This type of 
equipment is not illustrated or discussed in detail herein so as 
to avoid unnecessarily obfuscating the instant invention. For 
purposes of understanding the instant invention, it is suffi- 
cient to recognize that additional conventional equipment of 
this type may be useful in establishing and maintaining com- 
munications between the various users. 

At least two of the devices in the system 100 have software, 
such as an application program, installed thereon to allow an 
instant messaging session to be initiated and conducted. An 
instant messaging session may include real-time or near real- 
time communications. FIG. 3 illustrates a flowchart of a por- 
tion of the software associated with initiating the instant 
messaging session and controlling a graphical user interface 
(GUI) used by the participants to the instant messaging ses- 
sion. In particular, the process begins at block 300 in a con- 
ventional manner with one of the two parties sending the 
other party an invitation to initiate an instant messaging ses- 
sion. Assuming that the other party accepts the invitation, the 
software on each party’s computer initiates the GUI, which 
opens a window where both parties’ messages and other 
pertinent information and controls are displayed. An exem- 
plary representation of the GUI is shown in FIG. 4 and may be 
referenced simultaneous with the discussion of FIG. 3 herein 
for a more complete understanding of the operation of the 
instant invention. 

The messages exchanged by the participants may contain 
information regarding an icon to be used to represent each 
party. For example, party A may select an icon, such as 
“Mary” 400 as a graphical representation of party A. Party B 
may receive and store the icon and then display it adjacent a 
message delivered by party A. The icon makes it easier for 
party B to more quickly identify those messages associated 
with party A. An exemplary exchange of messages in which 
party A has sclected the icon “Mary” 400 and party B has 
selected the icon “Sue” 402 is shown in FIG. 4. Displaying 
unique graphical icons allows a user to readily identify the 
speaker with a quick glance. Additionally, displaying the 
icons adjacent cach party’s message allows the users to iden- 
tify the speaker without looking away from the message 
region of the GUI. In an alternative embodiment, the user may 
elect to display not only the icon, but also the name associated 
with the author of the message. On the other hand, the user 
may elect to display only the name associated with the author 
of the message, preventing the icon from being displayed 
altogether, if desired. 

Those skilled in the art will appreciate that the icons need 
not be delivered with each message. That is, party A may send 
an icon during the initial portion of the session, and party B 
will associate the icon with party A. store it locally. and then 
retrieve and display it each time a message is received from 
party A. Additionally, party A’s icon may be overridden 
locally by party B. That is, party B may elect to display a 
different icon adjacent party A’s messages, at least on the GUI 
viewed by party B. Party B may select any of a plurality of 
icons stored locally, and indicate through the local GUI, such 
as by pointing and clicking on various pull-down menus 
provided by the local GUI, that the selected icon should be 
used when displaying party A’s messages. 

The GUI may also use additional strategies to graphically 
differentiate the parties of the instant messaging session. For 
example, a sending party may send an indication of a color 
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scheme in which his/her messages should be displayed. The 
receiving party may then, at his/her discretion, display the 
messages from the sender in the requested color scheme. 

Alternatively, the receiving party may elect to override the 
sending parlies requested preference, and instead display 
each parly’s message in ils own distinct color. That is, party A, 
during an initialization phase, may indicate through the local 
GUI that any message received from party B should be dis- 
played with red letters and a white background, and that any 
messages generated by himself, should be displayed with a 
yellow background and black letters. In either case, the color 
distinction allows the party to visually determine the author of 
a message without the need to read and understand an iden- 
tifying name, such as is illustrated in the prior art at FIG. 1 
(e.g., BobbyD211). 

Allowing the sender to select the color and style, however, 
may lead to some confusion in the event that another partici- 
pant to the instant messaging sessions elects a similar style 
and/or font. Empowering the receiver of the message to over- 
ride locally the style and color choices indicated by the sender 
may help to alleviate any confusion. That is, the receiving 
party may elect to display the message with a different color 
and style than indicated by the sending party, at least on the 
GU] viewed by the receiving party. The receiving party may 


select any ofa plurality of colors and styles stored locally,and 2 


indicate through the local GU], such as by pointing and click- 
ing on various pull-down menus provided by the local GUI, 
that the selected color and style should be used when display- 
ing the received messages. Alternatively, the GUI may be 
programmed to automatically assign a different color to each 
participant. 

An additional graphical distinction may be accomplished 
by partitioning the GUI into spatially distinct regions and then 
directing the messages to a region based upon its authorship. 
For example, the exemplary GUI of FIG. 4 has been generally 
divided into two horizontal regions, a left region 404 and a 
right region 406. For example, all] messages generated by the 
local user (party A), represented by Mary 400, are displayed 
in the right region 406, and all messages generated by the 
remote user (party B), represented by Sue 402. are displayed 
in the lefi region 404, It should be appreciated that the assign- 
ment of left and right regions to parties A and B, respectively, 
may be reversed without departing from the spirit and scope 
of the instant invention. Moreover, it is anticipated that the 
various spatially distinct regions may overlap. That is, a mes- 
sage generated by Mary 400 may extend from the right region 
406 and at least partially into the left region 404. Similarly, a 
message generated by Sue 402 may extend from the left 
region 404 and at least partially into the right region 406. 
Thus, the messages may at least partially overlap, depending 
on the length of the messages. 

Further. depending upon the number of participants, it may 
be useful to define more than two spatially distinct regions. 
For example, where three participants are present. 1 may be 
useful to provide three horizontal regions. 

In the exemplary GU] of FIG. 4, the text of the messages 
associated with Sue 402 is displayed in the left region and is 
left justified. Similarly the text of the messages associated 
with Mary 400 is displayed in the right region and is right 
justified. Those skilled in the art will appreciate that other 
justification schemes may be used without departing from the 
spirit and scope of the instant invention. 

In one embodiment of the instant invention, the order in 
which the messages appear on the GUI generally corresponds 
to the order in which they were received. For example, in the 
illustrated embodiment of FIG. 4, each message is displayed 
below previously received messages so that the order of the 
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conversation is preserved, with older messages appearing 
nearer the top of the GUI and newer messages appearing 
nearer the bottom of the GUI. As the display region ofthe GUI 
fills, old messages are scrolled up and out of view. A user may, 
however, activate a scrolibar mechanism 408 using conven- 
tional point and click techniques to alter the portion of the 
conversation presented in the GUI. For example, the user may 
move the scrollbar mechanism 408 upward to view an older 
portion of the conversation, or downward to view a more 
recent portion of the conversation. 

To further enhance the readability and to provide further 
graphical identification of the author of each message appear- 
ing in the GU], each message may be displayed in a speech 
balloon 410. The balloon 410 includes a tail section 412, 
which generally extends toward the icon associated with the 
author of the message. For example, each message from the 
user identified by the icon Mary 400 appears in a balloon 410 
that has its tail section 412 extending generally toward the 
icon Mary 400. In the event that an icon is not associated with 
the author of the message, the tail section 412 is stil] useful to 
graphically illustrate the author. That is, since the GUI is 
divided into left and right horizontal regions, 404, 406 a 
speech balloon 410 located in the left horizontal region 404 
with its tail section 412 extending toward the left will still 
provide a graphical indication of the author (e.g., Sue 402 in 
the embodiment of FIG. 4). 

The size of the balloon 410 is controlled according to the 
length of the message. That is, the GUI receives a message, 
determines the length of the message, determines the size 
(e.g., based on the number of lines of text to be displayed) of 
the balloon 410 required to display the message, and then 
draws the balloon 410 with text in the appropriate horizontal 
portion of the GU] using the colors, style, and icon associated 
with the author of the message. A more detailed discussion of 
the sizing aspect of the speech balloons may be found below 
in conjunction with FIGS. 9 and 10. 

During an instant messaging session it is often useful to 
indicate when a remote party is preparing a message to be 
sent. For example, after party A sends a message requesting a 
response, it is useful to know if party B is preparing the 
requested response. Knowing that the other party is about to 
respond allows a more natural flow to the conversation. For 
example, if party B does not answer a question from party A 
in a timely manner, party A may send a second, related 
request. Party B, however, may promptly respond to the first 
request, leaving party A to guess at whether the response 
applies to the first request. the second request, or both. 

Accordingly, in the embodiment of the GU] shown in FIG. 
3, al block 302 the software determines whether a message is 
being generated, and in the event that a message is being 
prepared, the software at block 304 delivers a signal to the 
other party indicating that a message is being penerated. One 
method for determining whether a message is being yener- 
ated is for the remote terminal to monitor an input field for any 
characters having been entered. such as via the keyboard. and 
report back to the local terminal. For example, if the software 
detects that a keystroke has been entered or that emoticons 
have been selected, then it assumes that a message is being 
prepared to be sent to the other party, and the software sends 


0 asignal indicating that a possible responsive message is being, 


prepared. 

At block 306, the software checks for a signal from the 
other remote party indicating that a message is being pre- 
pared. If such.a signal is received, control transfers to block 
308 where the GUI is activated to produce a graphical indi- 
cation that a message is being prepared by the other party. An 
exemplary representation of the graphical indicator is shown 
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in the exemplary GUI of FIG. 5. For example, a “thought 
bubble,” such as is often used in comic strips to indicate that 
a character is thinking, is displayed along with the icon asso- 
ciated with the party who is preparing the message. In the 
exemplary embodiment of FIG. 5, a “thought bubble” 500 
provides a graphical indication that Sue 402 is currently pre- 
paring a message. For a number of reasons, the thought 
bubble 500 is particularly efficient for conveying the idea that 
the other party is preparing, a response. First, the thought 
bubble 500 appears in the GUI in the same general region that 
a message would be displayed. Second, thought bubbles are 
common graphical representations familiar to many users. 
Third, because the thought bubble 500 is graphically similar 
to, but easily distinguishable from, the speech balloon 410, 
the user may intuitively understand its function as a precursor 
to an actual message. Accordingly, even inexperienced users 
may readily understand the function and operation of the 
instant messaging system, and will be able to more quickly 
participate in an instant messaging session at a higher level of 
proficiency. 

In an alternative embodiment, incomplete or partial mes- 
sages are communicated to the recipient as an indication that 
a message is being prepared. In this alternative embodiment, 
the partial message is accompanied by a graphic indication 
that the message is not yet complete, such as by“... ”” The 
partial messages are then periodically updated as more of the 
message is produced by the sender. 

At block 310, the software checks to determine if a mes- 
sage has been received from the other party. If so, control 
transfers to block 312 where the software displays the text 
message (or emoticon, or the like) along with the icon asso- 
ciated with the author. In this instance, any corresponding 
thought bubble is replaced by the corresponding speech bal- 
loon and its accompanying text. In the illustrated embodi- 
ments of FIGS. 4 and 5, the messages received from the other, 
remote party are displayed on the left side 404 of a display 
window in the GUI. Additionally, the text message is pre- 
sented in a speech balloon 410 and is left justified to further 
enhance its association with the other, remote party. 

At block 314. the software checks to determine if the 
message being prepared by the local party is complete. If so, 
control transfers to block 316 and the software delivers the 
message over the network connection to the other party. ‘he 
message is then displayed in the speech balloon 410 in 
replacement of the thought balloon. Additionally, the soft- 
ware displays the text message (or cmoticon, or the like) 
along with the icon associated with the author in the local 
GUI. In the illustrated embodiments of FIGS. 4 and 5, the 
messages produced by the local party are displayed on the 


right side 406 of a display window in the GUI. Additionally, 5 


the text message is presented in a speech balloon 410 and is 
right justified to further enhance its association with the local 
partly. 

Those skilled in the art will appreciate that while the instant 


invention has been depicted in exemplary embodiments in. 5: 


which there are two participants to an instant messaging 
session, the instant invention may be readily employed in 
instant messaging sessions involving three or more partici- 
pants. In one embodiment, all locally generated messages are 
presented on the right side 406 of the display window in the 
GUI, and all remotely generated messages are presented on 
the left side 404 of the display window. Thus, where there are 
two or more remote participants, each of their messages are 
presented on the left side 404 of the local participant’s display 
window. In other embodiments, each remote participant’s 
messages could be displayed in a spatially distinct region 
from that of the other participants. For example, messages 
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from first, second and third remote participants could be 
displayed in first, second and third regions, respectively, 
wherein the first, second and third regions are spatially dis- 
tinct. In the embodiment shown in FIG. 5B, a third participant 
is assigned to a central region, with his/her icon appearing in 
the central region and the associated speech bubble extending 
generally therefrom. 

Additionally, while the embodiments described herein 
have been shown with the GUI divided into spatially distinct 
horizontal regions, it is envisioned that other embodiments 
may spatially distinguish the various participants in other 
manners without departing from the spirit and scope of the 
instant invention. For example, the various participants may 
be distinguished by dividing the GUI into spatially distinct 
vertical regions with each participant being assigned a verti- 
cal portion of the GUI. For example, the local participant may 
be assigned the top region of the GUI, and the remote partici- 
pant may be assigned the bottom region. Additional remote 
participants may be grouped together or, as described above, 
assigned an intermediate region, such as a vertical middle 
region. In a GUI with vertically distinct regions it may also be 
useful to allow the conversation to scroll horizontally, as 
opposed to the common vertical scrolling employed in many 
instant messaging systems. For example, more recent mes- 
sages may be displayed to the right of older messages, with 
older messages scrolling off the left side of the GU] as the 
conversation advances. 

If the messaging session is complete, such as by one or both 
of the parties logging off of the network or otherwise shutting 
down the software, then block 318 detects the ending of the 
session and transfers contro] out to another program respon- 
sible for a proper and orderly winding up of the program. 
Otherwise, if the instant messaging session continues, then 
control transfers back to block 302 where the process repeats. 

Turning now to FIG. 6, an alternative embodiment of at 
least a portion of the software shown in FIG. 3 is illustrated. 
In this embodiment of the software, provision is made to 
preserve the order of a conversation during those instances in 
whicha first party is preparing a response but the second party 
nevertheless sends a response before the first party completes 
and sends its response. For example, consider the instant 
messaging session shown in FIG. 7 to illustrate an out-of- 
order presentation of messages in a conversation. In the 
embodiment illustrated in FIG. 7, Sue 402 is in the process of 
preparing a response to a message 700 generated by Mary 
400. Accordingly, a thought bubble 702 is positioned adjacent 
the Sue icon 402 below the message 700. Mary 400, however, 
did not wait for Sue’s response, but sent a message 704. Thus, 
once Sue 402 completes and sends the response, the thought 
bubble 702 will be replaced by a speech balloon (not shown) 
containing the message. If the speech balloon (not shown) 
merely replaces the thought bubble with re-ordering, then thé 
conversation will appear to have occurred in the order 700- 
702-704. even though the speech bubble replacing the 
thought bubble 702 occurred after. not before the speech 
bubble 704. This out-of-sequence ordering may give rise to 
confusion, particularly where the participants read the flow of 
the conversation at a subsequent time. 

The flowchart of FIG. 6 illustrates one embodiment of a 
method useful in reordering the speech balloons so that they 
appear in the GUI in the order in which they actually 
occurred. Generally, the process set forth in FIG. 3 is substan- 
tially similar to that of FIG. 6, with the exception of blocks 
600, 602, and 604. Generally, the order of the speech balloons 
is maintained based upon the time that the message was 
completed. Thought bubbles, on the other hand, are ordered 
based upon the time that they were created and are subse- 
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quently replaced by a speech balloon. Because a thought 
bubble may be created well before the corresponding speech 
balloon is completed, it is possible for other parties to com- 
plete messages in the intervening time. Thus, when the cor- 
responding speech bubble is completed and replaces the cor- 
responding thought bubble, the order of the speech balloons 
may vary. 

At block 306, the software checks for a signal from the 
other remote party indicating that a message is being pre- 
pared. If such a signal is received, control transfers to block 
600 where the GUI is activated to produce a graphical indi- 
cation that a message is being prepared by the other party. The 
order in which the graphical indication is displayed is based 
upon the time that the thought bubble was created. The time 
that the thought bubble was created may be determined from 
atime stamp provided by the remote user who is in the process 
of preparing the message. 

Thereafter, at block 310, the software checks to determine 
if a message has been received from the other party. If so, 
control transfers to block 602 where the software displays the 
text message (or emoticon, or the like) along with the icon 
associated with the author. In this instance, any correspond- 
ing thought bubble is removed and replaced by the corre- 
sponding speech balloon and its accompanying text. How- 
ever, the speech balloon is ordered based upon the time 
completed. The time that the speech bubble was completed 
may be determined from a time stamp provided by the remote 
user who generated the message. 

Anexemplary instant messaging session that illustrates the 
results of this ordering policy is presented in FIGS. 7 and 8. In 
the embodiment illustrated inT'IG. 7, Sue 402 is in the process 
of preparing a response to a message 700 generated by Mary 
400. Accordingly, a thought bubble 702 is positioned adjacent 
the Sue icon 402 below the message 700. Mary 400, however, 
did not wait for Sue’s response, but sent a message 704. 
Because the speech balloons 700, 704 are ordered based on 
the time completed whereas the thought balloon 702 is 
ordered based on the time created, the order of the messages 
will remain as shown in FIG. 7, until the message from Sue 
402 is finally received. 

‘thus. as is shown in FIG. 8. a speech balloon 800 has 
replaced the thought bubble 702, but is located after (or 
below) the speech balloon 704, as the speech balloon 800 was 
completed after the speech balloon 704. In this manner, the 
actual order of the conversation is preserved. 

A substantially similar process occurs with respect to dis- 
playing speech balloons associated with the local user. For 
example. at block 314, the software checks to determine ifthe 
message being prepared by the local party is complete. Ifso, 


contro! transfers to block 604 and the software delivers the 5 


message over the network connection to the other party. The 
message is then displayed in a speech balloon in an order 
based on the time that the message was completed. 

Turning now to FIGS. 9 and 10. one exemplary method for 


formatting and sizing the speech balloon 410 is shown. Inone ; 


embodiment, a text system. such as a standard text system 
used in Mac OS X is used to produce the text of the message. 
The text system provides information regarding the size of the 
text message to the GUI. The GUI uses this size information 
to construct a speech balloon of an appropriate size to contain 
the message. Certain rules regarding the formatting of the 
speech balloon affect the size of the speech balloon. For 
example, in one embodiment, upper, lower, left and right 
margins are established. In the exemplary embodiment illus- 
trated in FIG. 9A, the margins are selected as follows: 
upper—3; lower—5; left——13; and right—6. The text of the 
message is required to be positioned within the speech bal- 
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loon and within these margins. Using these margins causes 
the speech balloon to fit tightly around the message, as illus- 
trated in FIG. 9B, so as to be aesthetically pleasing while not 
unnecessarily consuming large portions of the GUI. Reduc- 
ing the size of the speech balloons allows more messages to 
appear on the GU] at one time. 

The GU] uses the information regarding the size of the text 
message and the desired margins to produce a speech balloon 
of the appropriate size. The process involves dividing a tem- 
plate speech balloon into nine regions, such as is shown in 
FIG. 10A. The nine regions are comprised of four corners, left 
and right edges, top and bottom edges, and a central region. 


Initially, the margins are added to the rectangular area taken: 


up by the text to produce a destination rectangle having 
dimensions in which the balloon is to be drawn. The four 
corner regions are drawn directly into the corners of the 
destination rectangle without significant change to their 
shape or size. The top and bottom edges are tiled horizontally 
into rectangles of the appropriate width (and original height). 
Varying the width of the top and bottom edges has the desir- 
able effect of altering the horizontal dimension of the speech 
balloon. The left and right edges are tiled vertically into 
rectangles of the appropriate height (and original width) to 
produce a stretched or modified speech bubble 1000, as 
shown in FIG. 10B. It should be noted that the destination 
rectangle can be smaller than the original template image in 
either or both its vertical and horizontal dimension. For 
example, in the embodiment illustrated in FIG. 10B, the ver- 
tical dimension of the speech balloon is smaller that the 
vertical dimension of the template speech balloon of FIG. 
10A, and the horizontal dimension of the speech balloon is 
larger that the horizontal dimension of the template speech 
balloon of FIG. 10A. 

Once the speech balloon is appropriately sized, it is color 
filled according to the requirements of the sender or the 
recipient, as discussed above. Coloration and shading of the 
speech balloon is accomplished by alternative methodolo- 
gies. In one embodiment, custom artwork is provided for each 
color to produce a desired variation across the surface of the 
speech balloon. For example. the color may be varied so that 
the coloration is lighter adjacent the bottom edge of the 
speech balloon. ‘This coloration scheme has proven to be 
pleasing to users. providing the appearance of three dimen- 
sional qualitics to the speech balloon. Alternatively, rather 
than developing custom artwork for each possible color, the 
specch balloon may be filled uniformly with the desired color. 
Thereafter, standard template shadows may be overlayed to 
produce a similar three dimensional effect. 

Finally, while FIGS. 9 and 10 illustrate the speech balloon 
410 drawn with its tail extending leftward, so as to be used in 
the lefi horizontal region of the GUI, the same processes 
described herein may be applied in drawing the speech bal- 
loon 410 with its tail extending rightward. so as to be used in 
the right horizontal region of the GUI. In particular, when the 
speech balloon 410 with a nightward extending tail is 
required, the speech balloon 410 with the leftward extending 
tail is designed to be of the proper size for the message, and 
then the speech balloon 410 is flipped horizontally or rotated 
about a central vertical axis to produce the appropriately sized 
speech balloon 410 with a rightward extending tail. In this 
manner, an economy of software coding is realized, as only a 
single routine for generating speech balloon with either left- 
ward or rightward extending tails is required. 

The particular embodiments disclosed above are illustra- 
tive only, as the invention may be modified and practiced in 
different but equivalent manners apparent to those skilled in 
the art having the benefit of the teachings herein. Further- 
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more, no limitations are intended to the details of construction 
or design herein shown, other than as described in the claims 
below. It is therefore evident that the particular embodiments 
disclosed above may be altered or modified and all such 
variations are considered within the scope and spirit of the 
invention. Accordingly, the protection sought herein is as set 
forth in the claims below. 


What is claimed is: 
1. A method for displaying information related to a com- 
munication session, comprising: 
displaying information relating to data produced by a plu- 
rality of participants to the communication session on a 
first display unit, wherein the information produced by a 
first participant of the plurality of participants is dis- 
played at a first position on the first display unit; 

receiving data from at least one of the remaining partici- 
pants of the plurality of participants by the communica- 
tion session; and 
displaying information relating to data received from the at 
least one of the remaining participants of the plurality of 
participants, on the first display unit, wherein the infor- 
mation received from the at least one of the remaining 
participants of the plurality of participants is displayed 
at a separate position for each said remaining participant 
of the plurality of participants on the first display unit, 
wherein the respective separate positions relating to 
each of the plurality of participants are all non-random, 
horizontal-spatially distinct and chronologically 
arranged. 
2. A method, as set forth in claim 1, wherein the respective 
positions relating to the plurality of participants are permitted 
to at least partially overlap. 
3. A method, as set forth in claim 1, wherein: 
receiving data from the remaining participants of the plu- 
rality of participants includes receiving data from a sec- 
ond participant of the plurality of participants; 

displaying information relating to data produced by the 
first participant further comprises displaying informa- 
tion relating to data produced by the first participant in a 
right justified format; and 

displaying information relating to data received from the 

second participant further comprises displaying infor- 
mation relating to data received from the second partici- 
pant in a left justified format. 
4. A method, as set forth in claim 1, wherein: 
receiving data from the remaining participants of the plu- 
rality of participants includes receiving data from a sec- 
ond participant of the plurality of participants; 

displaying information relating to data produced by the 
first participant further comprises displaying informa- 
tion relating to data produced by the first participant ina 
left justified format: and 

displaying, information relating to data received from the 

second participant further comprises displaying inlor- 
mation relating to data received from the second partici- 
pant in a right justified format. 

5. A method, as set forth in claim 1, wherem: 

displaying information relating to data produced by the 

first participant further comprises displaying informa- 
tion relating to data produced by the first participant ina 
speech balloon. 

6. A method, as set forth in claim 1, wherein: 

receiving data from the remaining participants of the plu- 

rality of participants includes receiving data from a sec- 
ond participant of the plurality of participants; 
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displaying information relating to data received from the 
second participant further comprises displaying infor- 
mation relating to data received from the second partici- 
pant in a speech balloon. 
7. A method, as set forth in claim 1, further comprising: 
receiving data from a second participant of the plurality of 
participants; 
detecting data being entered by the second participant 
before the data is delivered to the first participant; and 
10. displaying information relating to data being entered by the 
second participant on the first display unit. 
8. A method, as set forth in claim 7, wherein: 
displaying information relating to data being entered by the 
second participant on the first display unit further com- 
prises displaying a thought bubble on the first display 
unit. 
9. A method, as set forth in claim 7, wherein: 
displaying information relating to data being entered by the 
second participant on the first display unit further com- 
prises displaying the thought bubble on the first display 
unit at a second position on the first display unit. 
10. A method, as set forth in claim 7, wherein: 
displaying information relating to data being entered by the 
second participant on the first display unit further com- 
prises displaying preselected graphical information on 
the first display unit at a second position on the first 
display unit. 
11. A method, as set forth in claim 7, wherein: 
displaying information relating to data being entered by the 
30 second participant on the first display unit further com- 
prises displaying preselected graphical information on 
the first display unit at a second position on the first 
display unit; and 

removing the preselected graphical information from the 

35 first display unit in response to receiving data from the 
second participant. 

12. A method, as set forth in claim 11, wherein: 

displaying preselected graphical information on the first 

display unit at the second position on the first display 
40 unit further comprises displaying a thought bubble at the 
second position on the first display unit. 

13.4 computer readable program storage device encoded 
with instructions that, when executed by a computer, per- 
forms a method, comprising: 

45 displaying information relating to data produced by a plu- 
rality of participants to the communication session on a 
first display unit, wherein the information produced by a 
first participant of the plurality of participants is dis- 
played at a first position on the first display unit; 

sos receiving data from at least one of the remaining partici- 
pants of the plurality of participants by the communica- 
tion session; and 

displaying information relating to data received from the at 
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least one of the remaining participants of the plurality of 
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participants, on the first display unit. wherein the infor- 
mation received from the at least one of the remaining 
participants of the plurality of participants is displayed 
at a separate position for each said remaining participant 
of the plurality of participants on the first display unit, 
6 wherein the respective separate positions relating, to 
each of the plurality of participants are all non-random, 
horizontal-spatially distinct and chronologically 
arranged. 
j4. A computer readable program storage device, as set 
65 forth in claim 13, wherein the respective positions relating to 
the plurality of participants are permitted to at least partially 
overlap. 
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15. An apparatus for displaying information related to a 
communication session, comprising: 
means for displaying information relating to data produced 
by a plurality of participants to the communication ses- 
sion on a first display unit, wherein the information 
produced by a first participant is displayed at a firs 
position on the first display unit; . 

means for receiving data from at least one of the remaining 
participants of the plurality of participants by the com- 
munication session; and 

means for displaying information relating to data received 

from the at least one of the remaining participants of the 
plurality of participants on the first display unit, wherein 
the information received from the at least one of the 
remaining participants of the plurality of participants is 
displayed at a separate position for each said remaining 
participant of the plurality of participants on the first 
display unit, wherein the respective separate positions 
relating to each of the plurality of participants are all 
non-random, horizontal-spatially distinct and chrono- 
logically arranged. 

16. An apparatus, as set forth in claim 15, wherein the 
respective positions relating to the plurality of participants are 
permitted to at least partially overlap. 

17. A graphical user interface for displaying information 
related to a communication session, comprising: 

a plurality of regions adapted to display messages from a 

plurality of participants to the instant messaging session; 

a first region of the plurality of regions adapted to display 

at least one message from a first participant to the instant 
messaging session; and 

the remaining regions of the plurality of regions adapted to 

display at least one message from the at least one of the 
remaining participants of the plurality of participants 
respectively to the instant messaging session, wherein 
the respective separate plurality of regions relating to 
each of the plurality of participants are all non-random, 
horizontal-spatially distinct and chronologically 
arranged. 

18. A graphical user interface. as set forth in claim 17, 
wherein the respective plurality of regions relating to the 
plurality of participants are permitted to at least partially 
overlap. 

19. A graphical user interface, as set forth in claim 17, 
wherein the first region is adapted to display an icon associ- 
ated with the first participant, and a second region of the 
plurality of regions is adapted to display an icon associated 
with a second participant of the plurality of participants. 

20. A graphical user interface, as set Jorth in claim 17, 


wherein the first region is adapted lo display text from the first 


participant in a speech balloon. 
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21. A graphical user interface, as set forth in claim 20, 
wherein the first region is adapted to display the text.in a left 
justified format. 

22. A graphical user interface, as set forth in claim 17, 
wherein a second region of the plurality of regions is adapted 
to display text from a second participant of the plurality of 
participants in a speech balloon in a right justified format. 

23. A graphical user interface, as set forth in claim 17, 
wherein the first region is adapted to display a thought bubble 
in response to receiving a signal from the first participant that 
a message is being prepared to be sent to a second participant 
of the plurality of participants. 

24. A graphical user interface, as set forth in claim 23, 
further comprising a second region of the plurality of regions, 
wherein the first and second regions are adapted to display the 
messages received from the first and second participants ver- 
tically in the order in which they were received. 

25. A graphical user interface, as set forth in claim 24, 
wherein the first and second regions are adapted to replace the 
thought bubble with a corresponding speech balloon when 
the speech balloon is received. 

26. A graphical user interface, as set forth in claim 25, 
wherein the first and second regions are adapted to display the 
corresponding speech balloon in a vertical position corre- 


_sponding to the order in which it was received regardless of 


the vertical position in which the thought bubble was dis- 
played. 

27. A system, comprising: a first terminal; and 

a second terminal, wherein the first and second terminals 
each include a graphical user interface for displaying 
information related to a communication session 
between the first and second terminals, the graphical 
user interface comprising: 

a plurality of regions adapted to display messages from a 
plurality of participants; 

a first region of the plurality of regions adapted to display 
at least one message from a first participant to the instant 
messaging session; and 

the remaining regions of the plurality of regions adapted to 
display at least one message from the at least one of the 
remaining participants of the plurality of participants to 
the inslant messaging session, wherein the respective 
separate plurality of regions relating to each of the plu- 
rality of participants are all non-random, horizontal- 
spatially distinct and chronologically arranged. 

28. A system, as set forth in claim 27, wherein the respec- 

tive plurality of regions relating to the plurality of participants 
are permitted to at least partially overlap. 
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